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EXECUTIVE  SUMMARY 


Description  of  the  Test 

The  test  contains  75  multiple-choice  questions  covering  six  major  concepts 
based  on  the  subject  matter  and  process  skills  components  of  the  Grade  9 
Science  core  curriculum.  The  number  of  questions  and  the  percentage  emphasis 
given  to  each  major  concept  are: 


Major  Concept  No. 

Matter  Occupies  Space 
Kinetic  Molecular  Theory 
Heat  and  Temperature 
Energy 

Atoms  and  Molecules 
Process  Skills 


of  Questions  Emphasis  (%) 


13 

17 

19 

26 

18 

24 

7 

9 

10 

13 

8 

11 

Eleven  of  the  75  questions  are  taken  from  the  Minister's  Advisory  Committee  on 
Student  Achievement  (MACOSA)  Science  Test,  administered  in  1978. 

Administration 

The  test  was  administered  on  June  11,  1985  to  29  639  students  enrolled  in 
Grade  9 Science. 

Results 

The  provincial  averages  in  raw  scores  for  each  major  concept  and  the  total 
test  are: 


Matter  Occupies  Space 

9.6 

Kinetic  Molecular  Theory 

13.1 

Heat  and  Temperature 

10.9 

Energy 

4.7 

Atoms  and  Molecules 

6.3 

Process  Skills 

5.0 

Total  Test 

49.6 

The  percentage  of  Grade  9 Science  students  who  correctly  answered  the  11 
MACOSA  questions  was  greater  than  it  was  in  1978.  The  1985  average  score  was 
62.4%  compared  with  the  1978  average  score  of  54.7%. 
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Chapter  1 


THE  ACHIEVEMENT  TESTING  PROGRAM 


The  Achievement  Testing  Program  provides  information  significant  at  the 
provincial  and  local  levels  about  student  knowledge,  understanding,  and  skills 
in  relation  to  program  objectives. 

The  achievement  tests  are  specific  to  the  program  of  studies  prescribed  by  the 
Minister  of  Education.  Curriculum  specifications  for  each  subject  area, 
provided  by  the  Curriculum  Branch  and  the  Language  Services  Branch  of  Alberta 
Education,  identify  the  major  content  areas,  the  specific  learning  objectives 
within  each  area,  and  the  emphasis  that  each  objective  is  to  receive.  The 
test  questions  reflect  these  curriculum  specifications. 

The  achievement  tests,  administered  on  a cyclical  basis,  are  in  four  subject 
areas:  language  arts,  social  studies,  mathematics  and  science;  and  at  three 

grade  levels:  3,  6 and  9.  In  1985,  achievement  tests  were  administered  in 
Grade  3 Language  Arts,  Grade  6 Social  Studies  and  Grade  9 Science. 

Following  the  achievement  test  administration  in  June  of  each  year,  the 
results  are  reported  to  each  school  jurisdiction.  These  district  profiles 
include  results  for  each  school  and  each  student.  Individual  statements  of 
results  are  not  issued  to  students. 

This  provincial  report  is  designed  to  assist  school  jurisdictions  in 
interpreting  their  achievement  test  results. 


Exemptions  from  the  Achievement  Testing  Program 

Under  normal  circumstances,  the  following  are  exempt  from  achievement  testing: 

• Students  participating  in  Special  Educational  programs 

• Students  in  classes  where  the  subject  being  tested  has  been  cycled  and 
taught  in  an  alternate  year 

• Students  in  classes  where  the  subject  being  tested  has  been  taught  in  an 
alternate  semester 

• Students  enrolled  in  English  as  a Second  Language  Programs 
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Chapter  2 


DESCRIPTION  OF  THE  TEST 


Test  Design  and  Development 

The  development  of  the  Grade  9 Science  Achievement  Test  progressed  through 
three  stages:  preparation  of  curriculum  specifications,  development  of 
questions  and  selection  of  questions  for  the  achievement  test. 

1.  Curriculum  Specifications 

The  Curriculum  Branch  prepared  curriculum  specifications  based  on  the 
Program  of  Studies  for  Junior  High  Schools.  These  specifications 
identify  the  major  content  areas  and  the  emphasis  each  is  to  receive  in 
the  program.  The  curriculum  specifications  were  distributed  to  all  school 
jurisdictions  in  the  province. 

2.  Development  of  Questions 

The  Student  Evaluation  Branch  of  Alberta  Education  prepared  the  test 
blueprint  based  on  the  emphases  identified  in  the  curriculum 
specifications.  Test  questions  were  developed  by  Grade  9 Science  teachers 
from  all  parts  of  the  province  under  the  supervision  of  the  Student 
Evaluation  Branch.  The  questions  were  field-tested  on  year-end  tests. 
Revisions  were  made  to  the  questions  on  the  basis  of  teacher 
recommendations  and  field-test  results. 

3.  Achievement  Test 

The  achievement  test  was  constructed  from  the  questions  that  best  reflect 
curricular  intent  and  test  design  requirements.  The  Test  Review  Committee 
reviewed  the  test  for  content  validity,  accuracy  and  technical  merit. 

Final  revisions,  based  on  the  recommendations  of  this  committee,  were  made 
to  the  test. 

Reporting  Categories 

The  75  multiple-choice  questions  on  the  test  are  grouped  according  to  six 
major  concepts:  matter  occupies  space,  kinetic  molecular  theory,  heat  and 

temperature,  energy,  atoms  and  molecules,  and  process  skills.  The  specific 
concepts  tested  within  each  major  concept  category  are  identified  in  Chapter  5. 

Questions  are  classified  according  to  three  cognitive  levels:  knowledge, 
application  and  understanding,  and  higher  mental  activities.  The 
classification  of  test  questions  according  to  cognitive  level  depends  on  the 
manner  in  which  the  content  has  been  covered  in  the  classroom.  A question 
that  is  an  application  question  for  one  class  may  be  a knowledge  question  for 
another.  Teachers  may  wish  to  examine  this  classification  to  determine 
whether  it  is  consistent  with  the  way  they  taught  the  material. 

The  classification  of  questions  according  to  major  concepts  and  cognitive 
levels  is  identified  in  the  test  blueprint,  presented  in  Table  1.  An 
explanation  of  cognitive  levels  is  given  in  the  appendix. 
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Test  Blueprint 


Table  1 


Classification  of  Test  Questions 


COGNITIVE 
MAJOR  LEVEL 
CONCEPT 

KNOWLEDGE 

APPLICATION 

AND 

UNDERSTANDING 

HIGHER 

MENTAL 

ACTIVITIES 

TEST 

EMPHASIS 

(%) 

9.1  Matter  occupies 
space 

1,2,8 

4,5,6,7,9,10, 

12,13 

3,11 

17 

9.2  Kinetic  molecular 
theory 

14,21,26, 

30,31 

15,17,18,19, 

20,22,23,24, 

25,28,32 

16,27,29 

26 

9 . 3 Heat  and 

temperature 

33,40 

34,35,36,37, 

38,39,41,43, 

44,45,46,47, 

49 

42,48,50 

24 

9.4  Energy 

51,57 

52,54,55,56 

53 

9 

9.5  Atoms  and 
molecules 

66 

58,59,62,63, 

64,65,67 

60,61 

13 

Process  skills 

70,75 

68,69,71,72, 

73 

74 

11 

TEST  EMPHASIS  {%) 

20 

64 

16 

100 
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Chapter  3 


ADMINISTRATION  OF  THE  TEST 


Test  Population 

All  Grade  9 students  were  expected  to  write  the  Science  Achievement  Test. 
Exemptions  from  writing  were  approved  by  superintendents  under  specific 
conditions . 

Administrative  Procedures 

Jurisdictions  were  requested  in  April  to  report  the  number  of  students 
enrolled  in  Grade  9 Science  in  each  school.  In  May,  letters  were  sent  to  the 
superintendents  and  principals  regarding  the  test  schedule,  procedures  for 
test  administration  and  requirements  for  returning  test  materials. 

Information  addressed  to  the  teachers  related  to  the  administration  of  the 
test  and  the  return  of  test  materials.  Jurisdictions  were  sent  the 
appropriate  number  of  tests  and  administration  instructions,  packaged 
according  to  school.  Immediately  after  the  test  was  administered,  teachers 
were  instructed  to  collect  all  test  booklets  and  answer  sheets  and  return  them 
to  the  principal  for  forwarding  to  school  board  offices.  School  Boards  were 
responsible  for  sending  the  test  booklets  and  answer  sheets  to  the  Student 
Evaluation  Branch. 

Staff  from  the  Regional  Offices  of  Education  supervised  the  administration  of 
the  test  in  private  schools. 


Standard-Setting 

While  provincial  averages  are  useful  for  comparing  the  scores  of  students  in  a 
particular  school  or  jurisdiction  with  overall  levels  of  achievement,  it  is 
not  possible  to  know  whether  the  students  in  the  province  did  as  well  as  they 
should  have.  A test  score  by  itself  has  limited  meaning  without  comparison  to 
a standard.  Tests  vary  in  difficulty:  a raw  score  of  25  out  of  50,  for 

example,  could  represent  very  high  achievement  on  one  test,  and  very  low 
achievement  on  another. 

To  establish  standards  that  allow  the  assessment  of  achievement  on  the  test, 
the  Student  Evaluation  Branch  asked  forty  experienced  Grade  9 Science  teachers 
to  examine  each  question  on  the  test  and  determine  the  expected  difficulty 
level  of  that  question  for  two  groups  of  students:  border-line  passing 

students  and  borderline  honors  students.  From  the  individual  question 
difficulty  levels,  the  overall  test  difficulty  levels  expected  for 
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these  two  groups  were  determined.  The  averages  of  the  test  difficulty  levels 
established  by  the  teachers  provided  the  standards  for  minimum  performance  and 
honors  performance  on  the  test.  For  the  Grade  9 Science  Achievement  Test  the 
standards  established  were  as  follows: 

To  meet  the  minimum  performance  level,  students  should  achieve  a raw 
score  of  38.0. 

To  meet  the  minimum  honors  performance  level,  students  should  achieve  a 
raw  score  of  64.0. 
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Chapter  4 


RESULTS 


The  report  is  based  on  the  results  for  the  29  639  students  from  public  and 
separate  schools. 

Total  Test 


Since  over  28  580  of  the  students  completed  the  test,  it  was  concluded  that 
sufficient  time  was  allowed  to  write  the  test.  Test  statistics  are  as  follows: 

Raw  Score  Percentage 

Provincial  Average  49.6  66.2 

Standard  Deviation  12.5  16.7 

KR-20  Coefficient  0.9111 

The  percentage  of  students  meeting  the  minimum  performance  level  of  achievement 
on  the  test  was  81.3%. 

The  percentage  of  students  meeting  the  minimum  honors  performance  level  of 
achievement  on  the  test  was  14.3%. 

Table  2 presents  total  raw  scores  as  well  as  the  relative  frequency  and  the 
cumulative  frequency  for  each  raw  score.  The  range  of  scores  was  from  6 to 
75.  Any  relative  frequency  or  any  cumulative  frequency  smaller  than  0.05  was 
rounded  to  0.0.  For  example,  5 students  or  0.02%  obtained  a score  of  13,  but 
the  relative  frequency  for  this  score  is  recorded  as  0.0. 
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Table  2 


Frequency  Distribution  of  Raw  Scores 


Raw 

Score 

Relative 

Frequency 

<%)* 

Cumulative 

Frequency 

(%)** 

Raw 

Score 

Relative 

Frequency 

<%>* 

Cumulative 

Frequency 

(%)** 

6 

0.0 

0.0 

43 

2.3 

31.2 

7 

0.0 

0.0 

44 

2.3 

33.5 

10 

0.0 

0.0 

45 

2.4 

35.9 

12 

0.0 

0.0 

46 

2.5 

38.5 

13 

0.0 

0.0 

47 

2.7 

41.2 

14 

0.0 

0.1 

48 

2.7 

43.9 

15 

0.1 

0.1 

49 

2.8 

46.8 

16 

0.1 

0.2 

50 

2.8 

49.5 

17 

0.1 

0.3 

51 

2.9 

52.4 

18 

0.1 

0.5 

52 

2.8 

55.1 

19 

0.3 

0.7 

53 

2.9 

58.0 

20 

0.2 

1.0 

54 

2.9 

60.9 

21 

0.3 

1.3 

55 

2.9 

63.8 

22 

0.4 

1.7 

56 

2.8 

66.6 

23 

0.4 

2.1 

57 

2.9 

69.5 

24 

0.6 

2.8 

58 

2.9 

72.4 

25 

0.7 

3.5 

59 

2.9 

75.2 

26 

0.7 

4.2 

60 

2.7 

77.9 

27 

0.8 

5.0 

61 

2.6 

80.5 

28 

1.0 

6.0 

62 

2.7 

83.2 

29 

1.0 

7.0 

63 

2.4 

85.7 

30 

1.1 

8.1 

64 

2.3 

88.0 

31 

1.3 

9.3 

65 

2.2 

90.2 

32 

1.3 

10.7 

66 

2.2 

92.3 

33 

1.4 

12.1 

67 

1.6 

93.9 

34 

1.5 

13.6 

68 

1.5 

95.4 

35 

1.5 

15.2 

69 

1.5 

96.9 

36 

1.6 

16.8 

70 

1.1 

98.0 

37 

1.9 

18.7 

71 

0.8 

98.7 

38 

1.8 

20.5 

72 

0.6 

99.4 

39 

2.2 

22.7 

73 

0.4 

99.7 

40 

1.9 

24.6 

74 

0.2 

100.0 

41 

2.0 

26.6 

75 

0.0 

100.0 

42 

2.3 

28.9 

*Relative  frequency:  the  percentage  of  students  who  obtained  each  score 

**Cumulative  frequency:  the  percentage  of  students  who  scored  at  or  below 

each  score 
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Reporting  Categories 


Table  3 gives  details  about  scores  in  the  reporting  categories.  Provincial 
averages  for  these  categories  and  for  the  total  test  were  computed  and  rounded 
to  one  decimal.  Consequently,  the  sum  of  the  averages  is  not  exactly  the  same 

as  the  average  for  the  total  test. 

Table  3 

Provincial  Averages  for  Major  Concepts, 

Cognitive  Levels  and  Total  Test 


Reporting  Category 

Number  of 
Questions 

Average  in 
Raw  Score 

Standard  Deviation 
of  Raw  Score 

Major  Concept 

Matter  Occupies  Space 

13 

9.6 

2.5 

Kinetic  Molecular  Theory 

19 

13.1 

3.7 

Heat  and  Temperature 

18 

10.9 

3.6 

Energy 

7 

4.7 

1.6 

Atoms  and  Molecules 

10 

6.3 

2.2 

Process  Skills 

8 

5.0 

1.9 

Cognitive  Level 

Knowledge 

15 

10.9 

2.8 

Application  & Understanding 

48 

31.2 

8.4 

Higher  Mental  Activities 

12 

7.6 

2.3 

Total  Test 

75 

49.6 

12.5 
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Results  for  Individual  Questions 


The  percentage  of  students  choosing  each  response  is  given  in  Table  4.  The 
correct  response  for  each  question  is  also  identified. 

Table  4 


Results  for  Individual  Questions 


Item 

Key 

Distribution  of 
Responses  in  %* 

Item 

Key 

Distribution  of 
Responses  in  %* 

A 

B 

C 

D 

A 

B 

C 

D 

1 

D 

3.8 

16.0 

5.3 

74.7 

39 

B 

5.5 

67.9 

5.8 

20.8 

2 

C 

1.6 

14.8 

80.8 

2.8 

40 

C 

12.1 

6.8 

76.3 

4.8 

3 

B 

16.4 

65.8 

12.4 

5.3 

41 

C 

22.0 

12.8 

47.5 

17.6 

4 

A 

80.8 

7.3 

5.2 

6.7 

42 

A 

42.9 

13.8 

29.7 

13.4 

5 

A 

79.7 

4.4 

13.5 

2.3 

43 

B 

13.3 

26.5 

8.6 

51.5 

6 

D 

11.7 

11.7 

19.3 

57.0 

44 

D 

8.6 

3.9 

3.8 

83.7 

7 

B 

7.5 

74.9 

6.2 

11.3 

45 

D 

12.5 

4.6 

10.6 

72.2 

8 

B 

6.2 

76.1 

4.8 

12.9 

46 

D 

3.4 

6.4 

17.8 

72.3 

9 

B 

23.3 

68.6 

6.0 

2.0 

47 

D 

10.7 

11.6 

5.1 

72.5 

10 

C 

3.2 

3.5 

86.5 

6.8 

48 

D 

2.8 

11.1 

37.9 

48.1 

11 

C 

8.9 

4.9 

72.1 

14.0 

49 

D 

2.3 

21.8 

2.2 

73.6 

12 

B 

17.1 

62.4 

5.9 

14.6 

50 

C 

31.3 

12.3 

37.3 

18.9 

13 

A 

83.6 

2.5 

8.9 

4.9 

51 

C 

18.2 

3.9 

71.2 

6.6 

14 

A 

59.2 

11.2 

15.1 

14.4 

52 

B 

14.6 

57.8 

11.8 

15.7 

15 

C 

3.9 

11.0 

74.1 

10.9 

53 

C 

3.9 

21.0 

69.3 

5.7 

16 

B 

2.1 

89.3 

1.9 

6.7 

54 

A 

68.9 

13.2 

9.8 

8.0 

17 

A 

65.9 

14.6 

9.7 

9.6 

55 

B 

33.8 

43.8 

14.0 

8.2 

18 

C 

16.9 

27.5 

47.6 

8.0 

56 

A 

81.8 

1.4 

3.6 

13.2 

19 

C 

3.5 

17.5 

74.0 

5.0 

57 

D 

9.2 

4.6 

6.7 

79.5 

20 

A 

67.7 

12.3 

9.8 

10.1 

58 

C 

5.2 

20.2 

70.8 

3.8 

21 

C 

12.3 

7.2 

77.2 

3.3 

59 

D 

8.8 

20.1 

6.8 

64.2 

22 

B 

8.7 

76.0 

4.3 

11.0 

60 

A 

59.5 

15.7 

20.7 

3.8 

23 

C 

17.1 

7.4 

67.3 

8.1 

61 

C 

5.8 

6.5 

84.1 

3.5 

24 

A 

72.0 

6.9 

2.7 

18.3 

62 

B 

22.9 

57.1 

11.6 

8.4 

25 

A 

87.9 

4.3 

3.6 

4.1 

63 

B 

12.7 

61.8 

12.4 

12.9 

26 

A 

86.5 

4.1 

6.5 

2.8 

64 

B 

18.4 

47.6 

12.5 

21.3 

27 

B 

8.4 

59.7 

25.9 

5.9 

65 

A 

58.8 

15.0 

9.1 

16.9 

28 

A 

37.7 

20.2 

16.4 

25.6 

66 

B 

7.9 

55.8 

19.6 

16.5 

29 

D 

26.1 

8.6 

2.5 

62.7 

67 

B 

27.7 

66.2 

3.1 

2.8 

30 

B 

8.8 

71.0 

9.8 

10.3 

68 

D 

27.9 

12.1 

6.4 

53.3 

31 

C 

10.5 

4.6 

69.4 

15.4 

69 

C 

28.7 

14.1 

44.2 

12.8 

32 

A 

69.3 

19.2 

5.4 

6.0 

70 

D 

16.9 

7.9 

14.5 

60.5 

33 

A 

66.7 

6.5 

22.6 

4.1 

71 

A 

80.0 

12.1 

6.5 

1.1 

34 

B 

6.7 

44.1 

24.4 

24.6 

72 

B 

22.2 

60.2 

9.1 

8.3 

35 

C 

5.6 

20.6 

68.8 

4.8 

73 

D 

8.0 

9.4 

27.2 

55.2 

36 

B 

4.7 

68.8 

14.5 

12.0 

74 

C 

13.9 

11.5 

64.9 

9.5 

37 

D 

9.6 

7.8 

20.8 

61.6 

75 

C 

3.4 

8.7 

82.2 

5.5 

38 

D 

7.3 

27.9 

10.2 

54.5 

*The  sum  of  the  percentages  for  each  question  may  be  less  than  100%  because 
the  No  Response  category  is  not  included.  The  No  Response  category  does  not 
exceed  1%  for  all  questions. 
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Chapter  5 


DISCUSSION  AND  CONCLUSIONS 


Reporting  Categories 

The  results  for  each  of  the  major  concept  categories  are  discussed  in  detail 
in  the  following  sections.  The  concepts  tested,  the  easiest  question  and  the 
most  difficult  question  within  each  category  are  noted.  Sample  questions  from 
the  test  are  provided.  For  each  sample  question,  the  curriculum  concept  and 
the  cognitive  level  are  identified.  The  asterisk  (*)  indicates  the  correct 
response  for  each  question,  and  the  percentage  of  students  who  selected  each 
alternative  is  given. 

Matter  Occupies  Space  (Questions  1 to  13) 

This  subtest  comprises  questions  related  to  the  following  concepts: 

9.1  Matter  occupies  space  and  has  mass. 

1.  Fundamental  to  the  process  of  science  is  the  establishment  of 
standards  for  making  measurements. 

2.  Matter  can  be  measured  by  determining  its  linear  dimensions,  surface 
area  and  volume. 

3.  Matter  can  be  measured  in  terms  of  its  mass  and  weight. 

4.  Density  is  a characteristic  property  of  any  given  sample  of  matter 
and  is,  therefore,  useful  for  identification  purposes. 

• The  average  score  for  the  13  questions  on  this  subtest  was  74.1%. 

• Question  10,  requiring  students  to  interpolate  from  a graph,  was  the  easiest 
(86.5%  answered  correctly). 

• Question  6,  requiring  students  to  use  the  slope  method  to  determine  the 
density  of  a liquid,  was  the  most  difficult  (57.0%  answered  correctly). 
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Sample  question: 

Question  3 
Concept  9.1.4 

Cognitive  Level  - Higher  Mental  Activities 

In  this  question,  students  are  required  to  analyse  the  observations  and 
determine  the  relative  masses  (hence  densities)  of  the  three  blocks. 


3.  A student  was  given  three  blocks  of  different  metals  that  were 

labelled  Q, 

R,  and  Z.  Each  block  had  a volume  of  20  cm3. 

By  using  an 

equal-arm  balance,  the  student  found  that  block  Q just 

balanced  blocks  R and  Z when  they  were  placed  together.  However, 

block  R was 

not  heavy  enough  to  balance  block  Z.  From  this 

information 

we  know  that  the  blocks,  in  order  of  increasing  density. 

are 

Student  Responses  (%) 

A.  Q,R,Z 

16.4 

*B . R,Z,Q 

65.8 

C.  Z ,R,Q 

12.4 

D.  R,Q,Z 

5.3 

Kinetic  Molecular  Theory  (Questions  14  to  32) 

This  subtest  comprises  questions  related  to  the  following  concepts: 

9.2  The  forms  and  behavior  of  matter  can  be  explained  by  the  Kinetic 
Molecular  Theory. 

1.  Matter  is  composed  of  tiny  particles. 

2.  Molecules  are  in  a state  of  constant  motion. 

3.  Molecular  motion  is  the  basis  for  diffusion 

4.  Molecular  motion  results  in  evaporation. 


• The  average  score  for  the  19  questions  on  this  subtest  was  69.2%. 

• Question  16,  requiring  students  to  formulate  a hypothesis,  was  the  easiest 
(89.3%  answered  correctly). 

• Question  28,  requiring  students  to  identify  the  manipulated  variable,  was 
the  most  difficult  (37.7%  answered  correctly). 
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Sample  questions: 


Use  the  following  information  to  answer  questions  27  to  29. 


Four  thermometer  bulbs  were  each  wrapped  in  a layer  of  cotton  cloth 
and  placed  into  separate  beakers  for  two  minutes. 

Thermometer  J was  placed  into  a beaker  containing  air.  Thermometer 
K was  placed  into  a beaker  containing  liquid  K.  Thermometer  L was 
placed  into  a beaker  containing  liquid  L.  Thermometer  M was  placed 
into  a beaker  containing  liquid  M. 

After  the  thermometers  were  removed  from  the  beakers,  the  temper- 
ature for  each  thermometer  was  recorded  every  minute  for  12  minutes. 

The  results  are  shown  on  the  graph. 


temperature 

(°C) 
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Question  27 

Concept  9.2.4d  - Different  liquids  evaporate  at  different  rates. 

Cognitive  Level  - Higher  Mental  Activities 

In  this  question,  students  are  required  to  analyse  the  experimental  design  and 
determine  the  appropriate  problem  statement. 


27.  What  is  the  problem  that  was  being  investigated  in  this  experiment? 

Student  Responses  (%) 


A.  What  effect  does  the  type  of  cloth  have  8.4 

on  the  rate  of  evaporation? 

*B.  What  effect  does  the  type  of  liquid  have  59.7 

on  the  rate  of  evaporation? 

C.  What  effect  does  the  air  temperature  have  25.9 

on  the  rate  of  evaporation? 

D.  What  effect  does  the  quantity  of  liquid  have  5.9 

on  the  rate  of  evaporation? 


Question  28 
Concept  9.2.4d 

Cognitive  Level  - Application  and  Understanding 

In  this  question,  students  are  required  to  identify  the  manipulated 
(independent)  variable  in  the  experiment. 


28.  The  manipulated  (independent) 

variable  in  this  experiment  is  the 

Student  Responses  (%) 

*A.  type  of  liquid 

37.7 

B.  air  temperature 

20.2 

C.  cotton  wrapping 

16.4 

D.  rate  of  evaporation 

25.6 

Question  29 
Concept  9.2. 4d 

Cognitive  Level  - Higher  Mental  Activities 

In  this  question,  students  are  required  to  infer  that  the  substance  having  the 
highest  rate  of  temperature  change  is  the  substance  having  the  highest  rate  of 
evaporation. 


29.  According  to  the  graph. 

the  substance  that  has  the  highest  rate  of 

evaporation  is 

Student  Responses  (%) 

A.  air 

26.1 

B.  liquid  K 

8.6 

C.  liquid  L 

2.5 

*D.  liquid  M 

62.7 
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Heat  and  Temperature  (Questions  33  to  50) 

This  subtest  comprises  questions  related  to  the  following  concepts: 

9.3  Heat  and  temperature  can  be  explained  in  terms  of  molecular  motion. 

1.  Heat  and  temperature  are  related. 

2.  Matter  exists  in  different  states. 

3.  A relationship  exists  between  molecular  motion  and  volume  occupied 


• The  average  score  for  the  18  questions  on  this  subtest  was  60.3%. 

• Question  44,  requiring  students  to  understand  how  surface  characteristics 
affect  the  ability  of  an  object  to  absorb  heat,  was  the  easiest  (83.7% 
answered  correctly) . 

• Question  43,  requiring  students  to  identify  the  liquid  having  the  highest 
freezing  temperature,  was  the  most  difficult  (26.5%  answered  correctly). 

Sample  question: 

Question  35 

Concept  9.3.1 

Cognitive  Level  - Application  and  Understanding 

In  this  question,  students  are  required  to  apply  the  algorithm  Q = mcAt. 


35.  How  much  heat  energy  is  required  to  raise  the  temperature  of  100  g 
of  water  from  15°C  to  30°C? 

Note:  Specific  heat  of  water  = 4200  J/kg*°C. 


by  matter 


Student  Responses  (%) 


A.  21  J 

B.  1500  J 
*C.  6300  J 

D.  26  460  J 


5.6 

20.6 


68.8 


4.8 
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Energy  (Questions  51  to  57) 

This  subtest  comprises  questions  related  to  the  following  concepts: 

9.4  Energy  enables  work  to  be  done  and  motion  to  be  changed. 

1.  Energy  can  be  described  as  either  kinetic  or  potential  energy. 

2.  Energy  is  present  in  the  universe  in  several  forms. 

3.  One  form  of  energy  can  be  changed  into  another. 

• The  average  score  for  the  7 questions  on  this  subtest  was  67.5%. 

• Question  56,  requiring  students  to  apply  the  principles  of  energy 
transformation,  was  the  easiest  (81.8%  answered  correctly). 

• Question  55,  requiring  students  to  identify  an  energy  transformation,  was  the 
most  difficult  (43.8%  answered  correctly). 

Sample  question: 

Question  51 
Concept  9.4.1 

Cognitive  Level  - Knowledge 

In  this  question,  students  are  required  to  recall  the  definition  of  potential 
energy. 


51.  The  energy  of  a mass  held  10  m above  the  ground  is  said  to  be 


Student  Responses  (%) 


A.  kinetic 

B.  chemical 
*C.  potential 

D.  mechanical 


18.2 

3.9 

71.2 

6.6 
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Atoms  and  Molecules  (Questions  58  to  67) 

The  subtest  comprises  questions  related  to  the  following  concepts: 

9.5  Matter  is  composed  of  atoms  and  molecules. 

1.  Theories  and/or  models  have  been  developed  to  assist  in 
understanding  atoms. 

2.  A relationship  exists  between  atoms  and  molecules. 

3.  A relationship  exists  among  elements,  compounds  and  mixtures. 

4.  There  is  a different  between  physical  and  chemical  change. 

• The  average  score  for  the  10  questions  on  this  subtest  was  62.6%. 

• Question  61,  requiring  students  to  apply  newly  discovered  knowledge,  was  the 
easiest  (84.1%  answered  correctly). 

• Question  64,  requiring  students  to  evaluate  the  usefulness  of  procedures, 
was  the  most  difficult  (47.6%  answered  correctly). 

Sample  question: 

Use  the  following  information  to  answer  question  63. 


Blue  copper  sulphate  turns  white  when  heated  because  water  is  driven  off. 
A student  obtained  the  following  results  when  heating  some  copper 
sulphate: 

Mass  of  beaker  48.7  g 

Mass  of  beaker  containing  unheated  copper  sulphate  319.0  g 

Mass  of  beaker  containing  heated  copper  sulphate  204.2  g 


Question  63 
Concept  9.5.1 

Cognitive  Level  - Application  and  Understanding 

In  this  question,  students  are  required  to  calculate  the  mass  of  water  in 
unheated  copper  sulphate  from  the  data. 


63.  What  was  the  mass  of  water 

in  the  unheated  copper  sulphate? 

Student  Responses  (%) 

A.  163.5  g 

12.7 

*B.  114.8  g 

61.8 

C.  67.1  g 

12.4 

D.  66.5  g 

12.9 
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Process  Skills  (Questions  68  to  75) 

While  the  process  skills  component  of  the  Grade  9 Science  curriculum  is 
integrated  with  the  subject  matter  component  throughout  the  test,  an  attempt 
was  made  in  this  subtest  to  isolate  the  process  skills  component. 

The  subtest  comprises  questions  related  to  the  following  process  skills: 

1.  Interpreting  data 

2.  Applying  newly  discovered  knowledge 

3.  Formulating  hypotheses 

4.  Controlling  variables 

5.  Designing  experiments 

• The  average  score  for  the  8 questions  on  this  subtest  was  62.6%. 

• Question  75,  requiring  students  to  select  the  definition  for  "hypothesis," 
was  the  easiest  (82.2%  answered  correctly). 

• Question  69,  requiring  students  to  compute  an  average  from  data  presented  in 
a bar  graph,  was  the  most  difficult  (44.2%  answered  correctly). 
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Sample  question: 


Use  the  following  information  to  answer  question  74. 


A Grade  9 Science  class  listed  the  variables  that  they  thought  might  affect 
the  time  required  to  melt  an  ice  cube.  The  list  included: 

mass  of  the  ice  cube 
temperature  of  the  room 
shape  of  the  ice  cube 

One  student  decided  to  test  the  hypothesis  that 
affects  the  time  required  to  melt  the  ice  cube. 

the  shape  of  an  ice  cube 

Question  74 

Process  Skill  - Designing  experiments 
Cognitive  Level  - Higher  Mental  Activities 

In  this  question,  students  are  required  to  evaluate  the  applicability  of  an 
experimental  design  to  the  hypothesis. 


74.  Which  design  should  the  student  select  to  test 

this  hypothesis?  Student  Responses  (%) 

A. 

Use  five  ice  cubes,  each  with  a different 
shape  and  mass.  Use  five  identical 
containers,  all  at  the  same  temperature. 
Observe  the  melting  time  of  the  ice  cubes. 

13.9 

B. 

Use  five  ice  cubes,  all  having  the  same 
shape,  but  each  having  a different  mass. 
Use  five  identical  containers,  all  at  the 
same  temperature.  Observe  the  melting 
time  of  the  ice  cubes. 

11.5 

*C. 

Use  five  ice  cubes,  all  having  the  same 
mass,  but  each  having  a different  shape. 
Use  five  identical  containers,  all  at  the 
same  temperature.  Observe  the  melting 
time  of  the  ice  cubes. 

64.9 

D. 

Use  five  ice  cubes,  all  having  the  same  mass, 
but  each  having  a different  shape.  Use  five 
identical  containers,  each  at  a different 
temperature.  Observe  the  melting  time  of  the 
ice  cubes. 

9.5 
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Comparison  With  the  1978  MACOSA  Results 


In  1978,  the  Minister's  Advisory  Committee  on  Student  Achievement  (MACOSA) 
Science  Test  was  administered  to  Grade  9 Science  students  in  Alberta.  Eleven 
questions  from  that  test  are  included  on  the  1985  Grade  9 Science  Achievement 
Test.  The  percentage  of  students  answering  each  of  the  common  questions 
correctly  and  the  means  for  the  two  administrations  are: 


:ion  Number 

1978  MACOSA  Results  (%) 

1985  Results  (%) 

2 

65.0 

80.8 

8 

77.1 

76.1 

46 

68.0 

72.3 

64 

43.9 

47.6 

65 

47.0 

58.8 

66 

50.0 

55.8 

68 

36.8 

53.3 

69 

32.4 

44.2 

72 

56.4 

60.2 

73 

54.0 

55.2 

75 

71.5 

82.2 

Mean 

54.7 

62.4 

Total  Test 


In  a comparison  of  process  skills  questions  across  the  total  test,  students 
had  more  difficulty  with  questions  that  required  them  to  identify  variables 
(18  and  28)  than  they  had  with  questions  that  required  them  to  identify  the 
problem  statement  or  hypothesis  (16,  27,  71  and  74). 

The  average  difficulty  levels  in  the  major  concept  categories  show  some 
variation  (across  the  total  test).  Though  an  attempt  was  made,  it  was 
impossible  to  select  sets  of  questions  so  that  the  average  difficulty  level  in 
one  category  was  equal  to  that  in  another.  Hence  the  variation  might  be  due 
to  variations  in  question  difficulty  rather  than  in  student  performance. 
Student  performance  in  each  category  approximated  the  expected  level. 

Because  questions  within  each  cognitive  level  category  vary  in  difficulty,  and 
because  the  average  difficulty  of  the  questions  in  each  category  should  not 
necessarily  be  equal,  no  discussion  about  student  performance  on  the  three 
categories  across  the  total  test  will  be  made. 


Conclusions 


The  provincial  average  for  the  total  test  was  66.2%.  The  percentage  of 
students  who  achieved  the  minimum  honors  level  of  performance  (raw  score  of 
64.0)  on  the  test  was  14.3%.  The  percentage  of  students  who  achieved  the 
minimum  level  of  performance  (raw  score  of  38.0)  on  the  test  was  81.3%. 
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Chapter  6 


GUIDE  TO  THE  INTERPRETATION  OF  JURISDICTION  RESULTS 

In  addition  to  their  use  in  monitoring  student  achievement  for  the  province  as 
a whole,  the  results  of  the  Grade  9 Science  Achievement  Test  are  useful  in 
comparing  achievement  in  a particular  jurisdiction  with  provincial  results. 
However,  care  must  be  exercised  in  making  these  comparisons  and  in  drawing 
conclusions  from  the  data. 

The  following  jurisdiction  and  school  reports  are  provided  under  separate 
cover  for  each  jurisdiction. 

1.  The  Jurisdiction  Summary  Report  contains  jurisdiction  equivalents  of  the 
provincial  results  that  are  given  in  all  statistical  tables  in  this  report 

2.  The  School  Summary  Report  contains  the  school  equivalents  of  the 
provincial  results  that  are  given  in  all  statistical  tables  in  this  report 

3.  The  Individual  Student  Subtest  Results  are  reported  for  each  school. 

These  reports  are  confidential  to  the  jurisdiction. 

Differences  Between  Jurisdiction  and  Provincial  Averages 

Jurisdictions  are  provided  with  their  average  scores  for  each  reporting 
category.  These  scores  may  be  compared  to  the  provincial  average  for  the  same 
reporting  category.  However,  the  importance  of  the  differences  between  group 
jurisdiction  averages  and  provincial  averages  is  not  always  clear.  To  aid  in 
the  interpretation  of  differences  between  the  averages,  jurisdiction  and 
school  reports  indicate  when  the  difference  is  unlikely  to  be  due  to  chance 
variation  in  the  abilities  of  students.  For  the  purposes  of  the  provincial 
testing  program,  the  95%  confidence  interval  is  used.  That  is  to  say,  if  the 
probability  is  less  than  1 in  20  that  the  difference  is  due  to  chance,  the 
difference  is  very  likely  a real  difference,  and  the  jurisdiction  average  is 
classified  as  different  from  the  provincial  average.  The  provincial  average 
for  that  reporting  category  determines  the  true  population  average.  The 
standard  deviation  for  the  jurisdiction  is  used  to  estimate  the  standard  error 
of  the  mean. 

Because  achievement  levels  are  calculated  taking  jurisdiction  size  into 
consideration,  two  jurisdictions  with  the  same  averages  but  of  different  sizes 
may  be  classified  differently.  The  larger  jurisdiction  would  be  more  likely 
to  be  above  or  below  average,  because  the  amount  of  chance  variation  would  be 
less  in  larger  jurisdictions,  and  the  actual  difference  would  represent  a 
larger  variation  from  the  provincial  average. 
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For  example,  imagine  two  jurisdictions,  A with  25  students  writing  Test  X,  and 
B with  100  students  writing  Test  X.  Both  jurisdictions  have  the  same  average, 
54.2;  both  jurisdictions  have  a standard  deviation  of  12.0.  The  difference 
between  the  provincial  average  and  the  jurisdiction  average  is  4.2.  A 
difference  this  large  would  be  expected  8 times  out  of  100  for  groups  of  25 
selected  at  random  from  the  population,  and  fewer  than  3 times  out  of  1000  for 
groups  of  100.  Thus  the  difference  from  the  provincial  average  would  not  be 
statistically  significant  for  Jurisdiction  A,  but  would  be  for  Jurisdiction  B. 

When  it  has  been  determined  that  a difference  is  significant,  the  direction  of 
the  difference  is  important,  particularly  for  those  jurisdictions  below  the 
provincial  average.  These  jurisdictions  are  encouraged  to  identify  the 
sources  of  these  differences. 

School  reports  contain  the  same  analyses  to  determine  whether  the  school 
varies  significantly  from  the  provincial  mean. 

Table  5 on  page  22  indicates  the  percentage  of  jurisdictions  classified  as 
significantly  above  or  below  the  provincial  average  for  each  reporting 
category. 
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Distribution  of  Jurisdiction  Levels  of  Achievement 


Table  5 indicates  the  percentage  of  jurisdictions  classified  as  significantly 
above  or  below  the  provincial  average  for  each  subtest. 

Table  5 

Distribution  of  Jurisdiction  Levels  of  Achievement 


Subtest 

Distribution  of  Jurisdictions 

X Below 
Provincial 
Average 

X Not  Different 
From  Provincial 
Average 

X Above 
Provincial 
Average 

Total  Test 

26.5 

57.7 

15.8 

Concepts 

Matter  Occupies  Space 

17.5 

62.9 

19.6 

Kinetic  Molecular  Theory 

36.0 

48.7 

15.2 

Heat  and  Temperature 

29.1 

56.1 

14.8 

Energy 

14.1 

65.6 

20.3 

Atoms  and  Molecules 

26.2 

56.4 

17.4 

Process  Skills 

22.3 

66.3 

11.4 

Cognitive  Levels 

Knowledge 

27.0 

57.1 

15.8 

Application  & Understanding 

27.9 

56.9 

15.2 

Higher  Mental  Activities 

21.4 

66.8 

11.7 

In  examining  the  test  results,  the  reader  must  keep  in  mind  that  a test  score 
does  not  indicate  why  a particular  performance  occurred,  but  only  that  it  did 
occur.  After  studying  the  results,  the  identification  of  reasons  for  that 
performance  should  be  undertaken.  There  are  a variety  of  factors  that  should 
be  examined: 

1.  Student  motivation.  Consideration  should  be  given  to  the  degree  to  which 
students  were  motivated  to  perform  to  their  levels  of  ability. 

2.  Student  ability.  While  the  statistical  test  of  significance  is  designed 
to  take  into  consideration  fluctuations  in  the  average  ability  levels  of 
students,  it  is  possible  that  a group  of  students  with  a particularly  high 
or  low  average  ability  may  come  through  a system.  This  is  much  more 
likely  to  be  a factor  in  small  systems  than  in  a large  one. 

3.  Readability.  The  achievement  test  was  designed  for  a Grade  9 reading 
level.  Jurisdictions  should  consider  the  average  reading  level  of  their 
Grade  9 students,  as  reading  levels  below  Grade  9 will  have  an  effect  on 
test  results  that  will  be  independent  of  achievement  in  science. 
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4.  Teaching  and  curriculum.  Consideration  should  be  given  to  the  type  of 

instruction  students  have  received  in  the  jurisdiction  and  the  adequacy  of 
curricular  implementation  of  the  program  elements  that  are  included  on  the 
test . 

There  will  be  other  factors  that  are  of  importance  in  particular 
jurisdictions.  School  boards  wishing  to  examine  further  the  results  in  light 
of  local  factors  are  encouraged  to  establish  their  own  local  interpretation 
panels . 

Absentee  Rates 

If  more  than  10%  of  the  eligible  students  in  a jurisdiction  did  not  write  the 
test,  the  reported  averages  for  that  jurisdiction  may  not  accurately  represent 
the  true  averages.  Teacher-assigned  marks  for  students  who  did  not  write 
could  be  compared  with  teacher-assigned  marks  for  students  who  did  write.  If 
the  averages  are  the  same  for  the  two  groups,  the  reported  achievement 
averages  are  probably  representative.  If  the  averages  are  different,  some 
estimates  can  be  made  of  what  the  achievement  averages  might  have  been  if  all 
students  had  written  the  test.  Jurisdictions  with  high  absentee  rates  may 
wish  to  contact  the  Student  Evaluation  Branch  for  assistance  in  estimating 
their  averages. 
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APPENDIX:  EXPLANATION  OF  COGNITIVE  LEVELS 


1.  Knowledge 

Knowledge  is  defined  as  including  those  behaviors  and  test  situations  that 
emphasize  the  remembrance,  either  by  recognition  or  recall,  of  ideas, 
material  or  phenomena.  This  level  comprises  knowledge  of  terminology, 
specific  facts  (dates,  events,  persons,  etc.),  conventions, 
classifications  and  categories,  methods  of  inquiry,  principles  and 
generalizations,  and  theories  and  structures. 

2.  Application  and  Understanding 

Application  requires  that  the  student  apply  an  appropriate  abstraction 
(theory,  principle,  idea,  methods)  to  a new  situation. 

Understanding  refers  to  responses  that  demonstrate  comprehension  of  the 
literal  message  contained  in  a communication.  This  means  that  the  student 
is  able  to  translate,  interpret  or  extrapolate.  Translation  refers  to  the 
ability  to  put  a communication  into  another  language.  Interpretation 
involves  the  reordering  of  ideas  (inferences,  generalizations  or 
summaries).  Extrapolation  is  the  ability  to  make  estimates  or  predictions 
based  on  an  understanding  of  trends  or  tendencies. 

3.  Higher  Mental  Activities 

Analysis,  synthesis  and  evaluation  are  included  in  the  category  of  higher 
mental  activities.  Analysis  comprises  the  ability  to  recognize  unstated 
assumptions,  to  distinguish  facts  from  hypotheses,  to  distinguish  a 
conclusion  from  statements  that  support  it,  to  recognize  facts  or 
assumptions  that  are  essential  to  a main  thesis  or  to  the  argument  in 
support  of  that  thesis,  to  distinguish  cause-effect  relationships  from 
other  sequential  relationships  and  to  recognize  the  point  of  view  of  a 
writer. 

Synthesis  is  the  production  of  a unique  communication,  the  ability  to 
propose  ways  of  testing  hypotheses  the  ability  to  design  an  experiment, 
the  ability  to  formulate  and  modify  hypotheses  and  the  ability  to  make 
generalizations. 

Evaluation  is  defined  as  making  judgments  about  the  value  of  ideas, 
solutions  and  methods.  It  involves  the  use  of  criteria  to  appraise  the 
extent  to  which  details  are  accurate,  effective,  economical  or 
satisfying.  Evaluation  includes  the  ability  to  apply  given  criteria  to 
judgments  of  work  done,  to  indicate  logical  fallacies  in  arguments  and  to 
compare  major  theories  and  generalizations. 
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